The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus capable of outputting an image having a wide dynamic range by using a solid state image device to synthesize and compress images of two or more screens having different quantities of exposure.
Hitherto, the upper limit for an image pickup dynamic range (a luminance range in which an image can be pick up) of an image pickup device, such as a television camera, is determined in accordance with the saturation level of the image pickup device. On the other hand, the lower limit of the same is determined in accordance with the noise level of the image pickup device and its peripheral circuits. The dynamic range of a usual image pickup device is about 50 dB to about 60 dB, while that of a TV (television) monitor or the like is about 45 dB. On the other hand, the dynamic range of a usual object is wider than the above-mentioned values and thus the same reaches 80 dB to 100 dB. Therefore, if a usual object is picked up by the image pickup device and the image of the object is displayed on a monitor or the like, a bright portion and a dark portion of an image having a contrast cannot simultaneously be displayed. Since the image pickup dynamic range is usually smaller than the luminance range for an object, a bright portion is whitened excessively and a low bright portion is blackened unsatisfactorily.
Accordingly, if a usual object is picked up by the image pickup device and the image of the object is displayed on a monitor or the like, a bright portion and a dark portion of an image having a contrast cannot simultaneously be displayed. In order to solve this problem, techniques have been disclosed in, for example, Jpn. Pat. Appln. KOKAI Publication No. 5-167889 and Jpn. Pat. Appln. KOKAI Publication No. 6-319149.
The apparatus disclosed in Jpn. Pat. Appln. KOKAI Publication No. 6-319149 has a structure such that the image of an object allowed to pass through an image pickup optical system is divided into two directions by a half mirror or the like. Then, a filter or the like is used to obtain, from the divided image signals, an image signal in which a dark portion of the object is deformed and a bright portion is not saturated and thus picked up satisfactorily is output and another image signal in which a bright portion is saturated and a dark portion is not deformed and thus picked up satisfactorily is obtained. The two image signals are synthesized so that an image signal having information from the dark portion to the bright portion can be obtained. The image signal is logarithmically compressed, and then subjected to a filtering process so that the dynamic range and the gain of the signal are controlled. Then, the image is output.
The apparatus disclosed in Jpn. Pat. Appln. KOKAI Publication No. 5-167889 has a structure such that an average value and a standard deviation of image data are used to calculate coefficients and add and multiply the coefficients so as to control the dynamic range and the gain. As a result of the foregoing method, an image having a wide latitude and excellent contrast can be displayed on the monitor.
Moreover, the following technique has been disclosed in Jpn. Pat. Appln. KOKAI Publication No. 57-39673: the shutter speed is controlled or an ND filter is used to change the quantity of exposure of the image pickup device, and then two obtained images having different quantities of exposure are synthesized so that the image pickup dynamic range is widened.
In Jpn. Pat. Appln. KOKAI Publication No. 7-75026, a technique has been disclosed in which the characteristic at a connection point of wide dynamic range signals obtained by synthesizing two or more images having different quantities of light are made to be adequate.
A technique is known which provides a knee characteristic as shown in FIG. 1 in order to compress a wide dynamic range signal.
However, the techniques disclosed in Jpn. Pat. Appln. KOKAI Publication No. 57-39673 and Jpn. Pat. Appln. KOKAI Publication No. 7-75026 have no contrivance to solve a problem in that the noise level at the connection point is rapidly changed. When two images having different quantities of exposure are synthesized, noise is simultaneously amplified because an image signal having a smaller quantity of exposure is multiplied with an exposure quantity ratio with the image signal having a larger quantity of exposure as shown in FIG. 2.
In an example case where images photographed with quantities of exposure respectively realized by shutter speeds of {fraction (1/60)} second and {fraction (1/2000)} second are synthesized, the signal obtained at the shutter speed of {fraction (1/2000)} second is amplified to 32 times. At this times, also the noise level is amplified to 32 times. Even if the dynamic range of the image to be picked up is not considerably wide, the noise level is amplified to 32 times. Therefore, the noise level is rapidly raised at the connection point and thus the quality of the image deteriorates excessively. The above-mentioned problems become more critical as the exposure quantity ratio is raised in order to widen the image pickup dynamic range.
The method for synthesizing two images having different quantities of exposure shown in FIG. 2 and arranged to provide the knee characteristic for the synthesized wide dynamic range signal is permitted to add two image signals having different quantities of exposure to realize the knee characteristic. It is preferably that two image signals having different quantities of exposure be switched because the S/N ratio deteriorates by about 3 dB. A case where the quantity of exposure is made such that the dynamic ranges are fixed to {fraction (1/60)} second and {fraction (1/2000)} second is considered. Since the dynamic range is fixed to {fraction (1/2000)} second even if the dynamic range of the object is not considerably wide and the adequate value of the high shutter speed is, for example, {fraction (1/200)} second, an image having a bright portion having an excessively low contrast is unintentionally formed.
The techniques disclosed in Jpn. Pat. Appln. KOKAI Publication No. 6-319149 and Jpn. Pat. Appln. KOKAI Publication No. 5-167889 have no contrivance to output a moving image in accordance with a calculation time period of the coefficients for controlling the dynamic range and the gain by calculating the average value and the standard deviation from image data. In a case where the brightness of the image is always changed as a moving image, a long time is required to perform the calculation of an image (for example, in a case where a light source, such as stroboscope is operated) having the brightness which is rapidly changed. Thus, there arises a problem in that an excessively long time is required to display an image having a wide latitude and excellent contrast.
Accordingly, a first object of the present invention is to provide an image pickup apparatus having a structure for enlarging the image pickup dynamic range by synthesizing a plurality of images having different quantities of exposure and capable of preventing rise of the noise level at the connection point.
A second object of the present invention is to provide an image pickup apparatus having a structure for enlarging the image pickup dynamic range by synthesizing a plurality of two images having different quantities of exposure and capable of preventing reduction in the contrast of bright portions to correspond to the dynamic range of an object required to be photographed.
A third object of the present invention is to provide an image processing apparatus capable of always outputting an optimum image even if the brightness is changed considerably.
That is, according to one aspect of the present invention, there is provided an image pickup apparatus comprising: image pickup means; detection means for detecting the level of an output signal from the image pickup means; determining means for determining a first quantity of exposure of the image pickup means in accordance with an output denoting a result of detection performed by the detection means; and synthesizing means for synthesizing a first image signal picked up with the first quantity of exposure and at least one second image signal picked up with a second quantity of exposure which is larger than the first quantity of exposure.
According to another aspect of the present invention, there is provided an image processing apparatus structured to pick up moving images of at least two screens such that the quantity of exposure is changed, synthesize picked up image data of at least two screens having the different quantities of exposure, compress the dynamic range of the synthesized image data to correspond to the dynamic range of an output apparatus and output the image, the image processing apparatus comprising: compression coefficient calculating means for calculating a first compression coefficient for performing compression of the dynamic range of synthesized image data; calculated compression-coefficient storage means for storing the first compression coefficient obtained by the compression coefficient calculating means; compression coefficient storage means for previously storing a second compression coefficient to be adaptable to a case where change in the brightness of an image exceeds a predetermined range when a moving image is picked up; compression coefficient output switch means for switching and outputting the first compression coefficient stored in the calculated compression-coefficient storage means in a case where the change in the brightness of the image when the moving image is picked up is included in the predetermined range and switching and outputting the second compression coefficient stored in the compression coefficient storage means in a case where the change in the brightness of the image when the moving image is picked up exceeds the predetermined range; and compressing means for compressing the dynamic range of image data obtained by synthesizing the images of at least two screens having different quantities of exposure in accordance with the compression coefficient output from the compression coefficient output switch means.
According to another aspect of the present invention, there is provided an image processing apparatus structured to pick up moving images of at least two screens such that the quantity of exposure is changed, synthesize picked up image data of at least two screens having the different quantities of exposure, compress the dynamic range of the synthesized image data to correspond to the dynamic range of an output apparatus and output the image, the image processing apparatus comprising: compression coefficient calculating means for calculating a first compression coefficient for performing compression of the dynamic range of synthesized image data; calculated compression-coefficient storage means for storing the compression coefficient obtained by the compression coefficient calculating means; compression coefficient setting means capable of setting and changing the compression coefficient stored in the calculated compression-coefficient storage means; compressing means for compressing the dynamic range of image data obtained by synthesizing the images of at least two screens having different quantities of exposure in accordance with the compression coefficient output from the calculated compression-coefficient storage means; and compression coefficient display means for displaying the compression coefficient output from the calculated compression-coefficient storage means.
Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out in the appended claims.